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ABSTRACT 

Taking into consideration the drivers’ state might be a serious challenge for designing new advanced driver 
assistance systems. During this paper we present a driver assistance system strongly coupled to the user. Driver 
Assistance by Augmented Reality for Intelligent Automotive is an augmented reality interface informed by a several 
sensors. Communicating the presence of pedestrians or bicyclists to vehicle drivers may end up in safer interactions 
with these vulnerable road users. Advanced knowledge about the presence of these users on the roadway is 
particularly important when their presence isn't expected or when these users are out of range of the advanced safety 
systems that are becoming a daily feature in vehicles today. For example, having advanced knowledge of a pedestrian 
walking along a rural roadway is important to increasing driver awareness through in-vehicle warning messages that 
provide an augmented version of the roadway ahead. Voice recognition system through an android platform adds 
some good flavour during this project. The strategy of voice recognition through this platform is achieved by 
converting the input voice signal into text of string and subsequently it's transmitted to embedded system which 
contains an arduino atmega328 microcontroller through Bluetooth as a technique of serial communication between an 
android application and a control system. The received text string on an arduino is also displayed on the AR Glass. As 
connected vehicles start to enter the market, it's conceivable that when the vehicle sensors detect a pedestrian on a 
rural roadway, the pedestrian presence is also communicated to vehicles upstream of the pedestrian location that 
haven't reached the destination. This paper presents a survey of studies related to perception and cognitive attention 
of drivers when this information is presented on Augmented Reality. 
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I. INTRODUCTION 
. ability of the drivers [1]. 


Driver Assistance Systems within the automotive industry 
are developing for several years, with a final goal to boost 
the security of the driving force and also the passengers, but 
also to enhance the performance, efficiency and luxury 
through ICT. The activity of operating a car is very complex. 
Partially, that's and therefore the the time limitations needed 
for the human to note, percept, process information before 
making to enhance, and act appropriately. This activity is 
occurring in an environment of traffic crammed with 
unpredictable situations. In such an environment, displaying 
appropriate information, like step-by-step navigation 
instructions, information about the road conditions, accidents 
or work on road, distance to the vehicle ahead, danger from 


The European Commission in its document the European 
Statement of Principles on Human Machine Interface for In- 
Vehicle Information and Communication Systems presents 
the framework recommendations and principles that require 
to be fulfilled when developing these components, further 
because the basic safety aspects that require to be followed. 
The document lists the key European directives and 
international standards that provide the framework for the 
manufacturers and suppliers of such equipment to be able to 
plan their development and implementation [2]. Geham 
(2005) states that so as to extend safety of the driving force 
and passengers it's more important for the vehicles through 


rockslides and similar information is highly useful as 
information in support of the choice making and preparation 
for appropriate action. problems with the ergonomic design 
aspect for these solutions are focused on determining the 
foremost appropriate ways for ensuring efficient and 
effective cooperation between human and also the system in 
context of that that the driving activity is of primary 
importance. The efficiency, or on the contrary, the negative 
effects of the aspect of road safety while using these systems 
is especially depended of the compatibility of their 


the systems for active safety to avoid accidents compared to 
the minimizing effects of the accidents through the systems 
for passive safety. The systems for driver assistance aim to 
extend the security and/or comfort and to help the driving 
force to specialize in the driving activity. 


Regarding the perception, it's been concluded almost 
immediately time that the driving performance are in tight 
correlation with the flexibility for seeing and visual strategy. 
Lot of authors comply with the actual fact that the perceptive 
visual channel is of paramount importance for the driving 
activity [3,4]. it's estimated that up to 90% of the needed 
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information for seamless completion of this task are 
communicated through the visual channel. the present 
systems in vehicles are mainly presenting visual messages: in 
a very variety of text messages, pictograms and/or graphic 
maps on displays integrated within the dashboard. When the 
driving force must operate these systems, he should move his 
view from the road for several seconds. The probability for 
an accident increases with the duration of the time the 
driving force isn't watching on the road [3, 4, 5]. Studies 
determine that the key duration of sight of the road while 
driving is 2 seconds [6]. In wider perspective, any integrated 
display within the vehicle is assessed as visual cost which 
will be quantified in a very number of occurrences and 
duration so as to induce an information from the system. 


Because there's a void or distance between the physical 
spaces (for example the road and also the vehicle interior) 
and also the virtual ICT spaces (for example the integrated 
display within the vehicle), the user should spend time and 
cognitive effort to regulate from one space to the opposite. 
This void is referred as cognitive distance between the 
physical and also the computer world [6]. Two separate 
components exist that consist the cognitive distance. the 
primary component is that the cognitive effort needed so as 
for the driving force to maneuver his/her attention from the 
physical to the pe environment and to locate the acceptable 
information therein space: moving the view from the road to 
the display. The second component is that the needed effort 
to return back from the pc environment to the physical world 
and to implement the gained information within the current 
activity like using the systems for GPS navigation, glance 
from the ICT map to the road and also the real environment 
and making a choice regarding the maneuvering and driving 
of the vehicle. The increased effort for completion of every 
of the components that make the cognitive distance lead to 
increased total results of the cognitive distance. Going into 
details, if the user should switch between spaces often the 
effect of the cognitive distance is even bigger. this is often 
significant for people with cognitive disabilities, those who 
complete activities that are in tight relation to the time 
duration, or activities that have big cognitive load. this is 
often especially important for older drivers that usually have 
weak cognitive system as results of their age. 


In this regard, the priority of the human factor is the way to 
define criteria for efficient recommendations for support 
within the design of the systems for driver assistance in 
vehicles with displaying visual messages without the 
necessity to distract the eye of the driving force from the 
driving activity [8]. this could also take into consideration 
the most important road questions of safety knowing that the 
amount of systems implemented in vehicles is getting bigger 
annually [9, 10]. 


II. RELATED WORK 


In recent decades, humans have relied on the car because the 
primary mode of transportation. Driving has become 
ubiquitous and is sort of use to most adults. However, the 
truth is that driving may be a complex task that needs 
processing large amounts of roadway and environmental 
information under tight time and pressure constraints. These 
constraints force drivers to prioritize information and to only 


process alittle percentage of visual auditory information 
without delay. Most of the time, drivers are able to handle 
the complexities of the driving process without mishaps. 
However, when drivers fail to react appropriately to a 
situation, or fail to acknowledge a dangerous scenario, the 
results of such an occurrence can range from a 
straightforward near-miss to a roadway fatality. Scenarios 
that involve a pedestrian fatality caused by drivers are often 
among those where the failure to acknowledge a dangerous 
situation is exemplified. 


A.Driver Behavior 


Drivers are liable to making mistakes thanks to inherent 
human physical, perceptual, and cognitive limitations. In 
fact, driver error has been identified to be the most cause in 
75 to 95% of roadway crashes [5, 6]. Human error has been 
the topic of research for a protracted time, resulting in 
several taxonomies that designate the speculation behind 
errors and supply a novel analysis perspective. However, 
three perspectives dominate: Norman’s [7] _ error 
categorization, Reason’s [8] slips, lapses, mistakes, and 
violations classification, and Rasmussen’s [9] skill, rule, and 
knowledge error classification [6]. These three perspectives 
categorize major driving errors into errors of recognition, 
errors of decision, and errors of performance. Perception 
and interpretation are often identified as recognition. 
Situations that may result in recognition errors include 
inattention, distraction, and looked-but-failed-to-see errors. 
Planning and intention are often identified as decision. 
Decision errors include misjudgment, false assumption, 
improper maneuver, excessive speed, inadequate signaling, 
and driving too near other vehicles. Action execution are 
often identified as performance. Performance errors include 
overcompensation, panic, freezing, and inadequate 
directional control . 


B. Augmented Reality Technology 

Augmented reality differs from virtual environments (VE), 
or video game because it is more commonly called. With 
VE technologies, users are completely immersed in an 
exceedingly synthetic environment and can't see the $64000 
world around them. In contrast, AR allows the users to work 
out the $64000 world with additional virtual objects overlaid 
on the $64000 world. Hence, instead of completely 
replacing reality, AR supplements it; virtual and real objects 
appear to coexist within the same space. the foremost 
commonly accepted definition of AR states that it's any 
system that has the subsequent three characteristics: 


1. It combines real and virtual; 
2. it's interactive in real time; and, 
3. it's registered in three dimensions 


Augmented reality is an example of intelligence 
amplification during which a computer is employed as a tool 
to create a task easier for somebody's to perform by 
providing additional information about the environment. It 
can provide drivers with a range of knowledge in an 
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exceedingly discrete manner and may be as simple as a 
further piece of knowledge or as complex as an entire 
environment makeover. Augmented reality relieves the 
burden on drivers by projecting what they see with 
informative details. It also improves the user’s perception 
and interaction with the $64000 world. Information that the 
user cannot easily or directly detect together with his or her 
own senses are indicated or emphasized by the virtual 
objects. This information helps the user better perform the 
real-world tasks 

With enhancements in special effects and also the increase 
in processing power of computers, AR technology has 
achieved a big jump. it's permeated several areas, like 
medical visualization and training, manufacturing and 
assembling, maintenance and construction, design and 
modeling, grooming and warfare, commercial applications, 
various kinds of entertainment, navigation, and data 
guidance. one in every of the foremost computationally 
intensive challenges that each one AR applications have in 
common is that the requirement to exactly align virtual 
images with objects within the world 


C.Testing In-Vehicle Technologies 


As the marketplace for in-vehicle technology continues to 
grow, there are several studies that involved in-vehicle 
technology. In-vehicle technology includes Bluetooth to 
voice command systems and data display systems. Although 
in-vehicle systems are useful in one or more ways, avoiding 
distraction and a spotlight deficits while driving may be a 
challenge in in-vehicle systems. Bach et al. [32] checked out 
100 papers and classified them into two categories: 
evaluation of settings for in-vehicle systems and measure of 
driver attention relevant to invehicle systems. The 
classification showed that the majority studies were 
conducted in driving simulators and real traffic driving. 
Lateral and longitudinal control and eye behavior were the 
most-used measures for driver attention. The studies also 
showed that in-vehicle systems interaction can increase 
safety while driving if minimal or no interaction induced 
visual demands. 


Using a driving simulator, Boyle and Mannering [30] 
evaluated driving behavior using in-vehicle and out-of- 
vehicle traffic advisory systems. Four different advisory- 
information conditions were analyzed: in-vehicle messages, 
out-of-vehicle messages, both kinds of messages, and no 
messages. Two weather scenarios were also considered, fog 
and no fog, also as two kinds of incidents, snowplows and 
no snowplows. Study results showed no _ significant 
difference in mean speed and variance speed over long 
segments. The study also found that when the knowledge 
message had either passed or become out of range, drivers 
would speed up to atone for the lost time incurred from 
being warned to abate. 


Ill. PROPOSED APPROACH 


Our objective is to integrate the visualization metaphor 
within the design process of driving assistance system. so as 


to propose a visualization metaphor adapted to the driving 
situation, it's necessary to grasp what's the present situation 
and what the driving force looks at. So we propose the 
coupling of an obstacle detection module with a system 
monitoring the driving force so as to develop a display 
module for an application of Augmented Reality 
(AR). 
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Fig 2:Block Diagram of Vehicle Unit 
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Having in mind the benefits of the systems and so as to 
check the consequences, differing types of HUD-AR 
displays are analyzed from the aspect of several of their 
functions for driver assistance. 


A. Lane Departure 

While testing the AR systems that present safe corridor for 
lane departure when driving so as to supply the motive force 
with the power to securely overtake a vehicle before of, the 
researchers have noticed a major improvement in two 
positive aspects: bigger number of the drivers used the 
braking pedal to lower the speed, which generally could be a 
positive indicator from the aspect of safety; all drivers 
operated the vehicle and braked in an exceedingly similar 
way, according the instructions of the required path. 
Nevertheless, the behavior of the drivers in adverse 
situations haven't been investigated, that's when the vehicle 
is within the dead spot or when overtaken by a faster 
vehicle. Furthermore, this study shows that in situations of 
lane departure, AAR systems have the tendency to form the 
drivers glance at the side rear view mirror later compared to 
the drivers not using such a system, because the visual 
attention of the motive force is firstly occupied by the AR 
display on the road (Figure 1.). After they interpret this AR 
information, the drivers checked the side mirror to organize 
for lane departure. 
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Fig 3: Driver Assistance System for lane departure 


Keeping the specified path of the vehicle while driving is 
especially difficult for unexperienced drivers and/or in 
atmospheric condition conditions when the visibility is 
lower. The concept of augmented reality enables outlining 
of the road edge with a virtual element assisting the driving 
force within the task of maneuvering the vehicle (Figure 2, 
Figure 3). By displaying a path for driving in augmented 
reality, the maintaining of the vehicle within the desired 
path of motion is achieved while lowering the deviations 
from the specified trajectory 


B. Detection of critical events on the road 

Drivers must use caution on the vehicles around them, the 
risks on the road, the specified path, pedestrians and traffic 
signs and every one of that while driving the vehicle, 
controlling its speed and direction. of these tasks increase 
the physical and therefore the mental workload, which is 
particularly dangerous for older drivers and drivers with 
lower reflexes. Hence, an alarm that might warn the driving 
force for an eminent danger on the road can assist in 
minimizing the workload of the driving force and reduce the 
quantity of accidents. the actual fact that a critical event is 
presented on the windshield can assist the driving force in 
detecting the damaging events (Figure 4, Figure 5). 
Compared to traditional systems for driver assistance, HUD- 
AR systems lower the time needed for detection of a 
happening up to 100ms. 


Fig. 4 Driver Assistance System for critical events detection 


C. Night Vision 


The systems for displaying information with AR can 
significantly improve the visualization in dark, emphasizing 
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the situation of pedestrians and other obstacles on the road, 
enabling drivers to efficiently transfer information that's 
instantly understandable. Night vision systems are known 
from time ago and for the primary time were employed in 
the military industry. within the automotive industry, this 
sort of systems was introduced for the primary time in a 
very serial production model by Mercedes-Benz in 2011. 
this method shows a picture on display placed within the 
electrical device of the vehicle. meaning that the driving 
force must move the eyes from the road, to interpret the 
image on the display, to return the view on the road and to 
implement the knowledge gained from the system. The 
systems for visual sense using AR present the exact same 
information but now directly on the windshield and 
therewith significantly lowering the workload of the driving 
force and decreasing the time needed for processing of the 
knowledge and taking action. 


D. Android Application 

Voice Control is an android application which recognize 
the speech of human then convert it into text format and this 
text as a string are often further proceed and transmitted 
from smart phone through Bluetooth as a wireless channel 
to an arduino Uno circuit where Bluetooth module HC-05 is 
installed for serial communication. during _ this 
communication system data is transferred one bit at a time. 
An arduino will receive the serial data and check the 
received string with predefined string and if it's equal then 
the corresponding operations are performed through digital 
pins of an arduino. MIT app inventor 2 is employed to 
develop this android application, which allows easy creation 
via drag and drop block programming. This platform is very 
beneficial to people who don't seem to be expert in java, 
android additionally as python coding. When the system 
Bluetooth is connected with android application then it 
displays the status “Connected” on the screen, then clicking 
the microphone button exposes the speech recognizer. It 
allows a straightforward and reliable reference to the Google 
speech Processing Libraries for smooth and accurate speech 
recognition. First we've to put in this application in our 
smart phone and connect the Bluetooth of our smart phone 
with system (HC-05 module) and switch on the mobile data 
then press the microphone button of an android application 
and speak something which convert immediately into text 
within the screen. 
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Fig 5: .Output for voice to text conversion 


IV. RESULTS EVALUATION 


Questionnaires were wont to gather subjective ratings as 
how they perceived the AR-HUD and the way they felt 
during the experiment. Out of 23 subjects in total, 15 
persons had a preference for AR-HUD with tracking 
conditions (65.5%), 4 of them had a preference for the none- 
tracked system and also the last 4 didn’t have any 
preference. The ARHUD with tracking conditions had 15 as 
median grade with a typical deviation of 6.5 (average = 
~13), and also the none-tracked system 10 median grade 
with a typical deviation of two.5 (average= ~11). 


Let’s start by looking further into the 15 people who had a 
preference for the AR-HUD with Tracking Conditions. the 
primary question that rises, is whether or not they need seen 
the difference all the way or only in specific situations. The 
histogram below shows the share of persons that have seen/ 
haven't seen the difference between the 2 systems in 
numerous road situations. 
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Fig 6:Different Road Situation 


After that, we wanted to look further into the scale factor in 
the case where the difference was seen between the two 
systems. In the graph below, we can see the number of 
persons with respective ranks for each specific road 
situation. The grades go from 5 (weak difference) to 20 
(strong difference). 
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Fig 7:Different Grade Scale 


For the 65.5% people that preferred the HUD with Tracking, 
our initial hypothesis was that the precision difference 
would be significant between the two systems and that 
imprecision ranks would be different in all of the following 
situations: intersections, curves, straight lines, overtaking 
manoeuvers, secondary tasks, and all the way in general . In 
order to measure this difference we used “Wilcoxon 
Statistical Test”, which is a non-parametric statistical test. In 
our case, we used the matched data for two tasks (driving 
session with tracking and driving session without tracking). 
As mentioned above, a large majority preferred the AR- 
HUD with tracking conditions, and those participants 
evaluated the system to be more precise in all different road 
situations than the other system. It was surprising that some 
persons (8 of them) didn’t manage to see the difference 
between the two systems. On closer examination, it was 
found that the majority of these drivers left a significant gap 
with the front vehicle, so they either had no preference or 
preferred the second configuration system. 


V. CONCLUSION 


Besides the actual fact that the HUD systems are present 
since the 80s, they're still not a usual way of displaying 
visual information within the automotive industry. Studies 
have shown that HUD displays have bigger potential, but 
they need lower acceptance level from drivers. one in every 
of the possible reasons for that why HUD systems are still 
not well established is that the incontrovertible fact that to 
this point the main target was on their development as 
technology, and not on adjusting to the requirements of their 
use by drivers. we introduce the event of supported Android 
Bluetooth Chatting for Smart Home Application. this 
technique is employed to regulate the electrical devices by 
low cost Bluetooth wireless technology without human 
intervention. the look principals of the classic 2D displays 
aren't applicable any further fully for this manner of 
presentation, due to the various habits of movement of the 
visualized objects. Further research is required within the 
area of determining the mixture of design principals that 
provide best results for a particular driving activity 
especially for HUD and AR. 


Based on that perspective, the review presented during this 
paper shows that HUD-AR visual displays have great 
potential from the aspect of driver assistance within the way 
of skyrocketing the perception and decreasing the work 
load, but with caution to the look principles and 
implementation of knowledge to the windshield. 
additionally, it's necessary to create additional studies in real 
conditions and not with the employment of driving 
simulator so as to induce better understanding of the 
acceptance level of the driving force for the HUD systems 
and to induce understanding of that where drivers wish to 
receive the data. 
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Further research should be conducted on the aspect of 
human factors so as to totally understand the ways of 
optimization of the massive technological advantages of the 
HUD-AR concept within the automotive industry with a 
final goal to extend the road safety. Understanding the 
challenges that these systems are bringing and their effect to 
the road safety within the everyday use in traffic should 
become a part of the capacities of the opposite stakeholders 
(police, departments for motorized vehicles, insurance 
experts, prosecutors and others) involved within the process 
of keeping the roads safe. 
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